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The MAILING DATE of this communication appears on the cover sheet with the correspondence address -- 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 03 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 7/23/02 ( No. 7) and 9/10/03 (No. 10) . 
2a)E3 This action is FINAL. 2b)Q This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-40 is/are pending in the application. 

4a) Of the above claim(s) 1-40 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-16 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 23 December 1998 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)n All b)D Some*c)Q None of: 

1 .D Certified copies of the priority documents have been received. 

2.0 Certified copies of the priority documents have been received in Application No. . 

3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) ^ Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1 .78. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) Q Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) Q Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 

U.S. Patent and Trademark Office 

PTOL-326 (Rev. 1 1 -03) Office Action Summary Part of Paper No. 1 1 
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DETAILED ACTION 



Response to Arguments 



1. Applicant's arguments with respect to claims 1-16 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claims 17-40 are withdrawn as indicated in paper No. 10 filed on September 10, 2003. 



2. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claim 5 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

In claim 5, lines 6-7, the phrase "of the at least one each story segment" is unclear. 
Appropriate correction is required. 



Claim Rejections - 35 USC §112 
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Claim Rejections - 35 USC § 103 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-11, 13-16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over "Broadcast News Navigation using Story Segmentation," by Andrew Merlino et al. 
(hereinafter referred to as Merlino), and in view of "Automatic Parsing of News Video," 
by HongJiang Zhang et al (hereinafter referred to as Zhang). 



Regarding claim 1, Merlino discloses a system that captures imagery (MPEG), audio 
(MPA) and closed-caption information. With MPG file, scene change detection and 
video classification (i.e., black frame, logo, etc.) is performed. Period of silence are 
detected from the MPA file. With the closed caption file, named entity tagging and token 
detection is performed. With all of this information, the process is performed to detect 
stories. With each detected story segment, a theme, gist and key frame is automatically 
generated and stored in the multimedia database (see page 386, section 4.1+). 
Inherently, the system comprising: a story segment identifier for processing a video 
stream (news stories) and partitioning the video stream into a plurality of story 
segments, the story segment identifier producing one or more key frames associated 
with each story segment of the plurality of story segments; Merlino discloses generating 
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gist, theme for each detected story segment (page 386, section 4.1, lines 17-18). 
However, Merlino does not specifically disclose associating one or more classifications 
to each story segment of the plurality story segments, to facilitate a selection among the 
plurality of story segments based on the one or more classification. 
Zhang discloses automatic parsing of News Video wherein the news video is 
partitioned into individual video shots; the partitioned video shots are then classified into 
anchorperson shots and news shots (see page 47, section 2.2 +). Necessarily, the 
system comprising a classifier for associating one or more classifications to each story 
segment of the plurality story segments, to facilitate a selection among the plurality of 
story segments based on the one or more classification. Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was make to modify 
Merlino to use the teaching as taught by Zhang in order to provide an easily selection 
between plurality of story segments. 

Regarding claims 2, Merlino teaches the video stream includes an associated text 
stream (closed caption information); the story segment identifier partitions the text 
stream into an at least one text segment corresponding to at least one story segment of 
the plurality of story segments (page 386, section 4.1 , lines 17-18). 
Zhang teaches the classifier associates the one or more classifications to the at least 
one story segment based on the at least one text segment (page 47, section 2.2+). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
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invention was make to modify Merlino to use the teaching as taught by Zhang in order 
to provide an easily selection among plurality of story segments. 

Regarding claims 3, Merlino teaches the video stream includes an associated audio 
stream (MPA); the story segment identifier partitions the audio stream into an at least 
one audio segment corresponding to at least one story segment of the plurality of story 
segments (page 386, section 4.1, lines 17-18). 

Zhang teaches the classifier associates the one or more classifications to the at least 
one story segment based on the at least one audio segment (page 47, section 2.2+). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was make to modify Merlino to use the teaching as taught by Zhang in order 
to provide an easily selection among^plurality of story segments. - 

Regarding claim 4, Merlino in view of Zhang teaches a system as discussed in the 
rejection of claim 3. However, neither Merlino nor Zhang specifically disclose converting 
video segment into text segment. Official Notice is taken that converting audio data to 
text data is well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art to modify Merlino and Zhang to use a well-known teaching in the 
art in order to helps impaired hearing viewers to understand the story easily. 

Regarding claim 5, Zhang discloses context information, such as news icon on screen, 
is also used in classification process (see figures 2-3 and section 2.2.1 +). Necessarily, 
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the system includes a visual characterizer, operably coupled to the story segment 
identifier and the classifier, for providing a visual characterization of at least one story 
segment of the plurality of story segments based on an image content of the at least 
one story segment; and wherein the classifier associates the one or more classifications 
to the at least one story segment based on the visual characterization (anchorperson, 
news program title bar, anchorperson name bar, background, news icon, etc.) 

Regarding claim 6, Zhang teaches the visual characterizer includes: a figure recognizer 
for recognizing a recognized figure (anchorperson, title bar, etc.) from a plurality of 
recognizable figures based on the image content; and wherein the visual characterizer 
characterizes the at least one story segment based on the recognized figure (figures 2-3 
and section 2.2+). _ . . _ _. 

Regarding claim 7, Zhang teaches the visual characterizer includes figure recognizer 
(section 2.2. 1+). 

Regarding claim 8, Merlino teaches the story segment identifier partitions the video 
stream based on at least one of a recognized figured, a recognized scene, a video cut, 
and a detected commercial (page 385, section 3.1.5+). 

Regarding claim 9, Zhang teaches the classification include at least one of: program 
type, news type, media, person, locale, popularity, and keyword (page 47, section 2.2+). 
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Regarding claim 10, Merlino teaches each story segment of the plurality of story 
segments include one or more scenes, and the one or more key frames correspond to a 
frame within each of the one or more scenes (page 386, section 4.1+ and figures 
4.2.2). 

Regarding claim 1 1 , Merlino teaches the key frame is determined based upon a 
transformed of an encoding of the story segment of the plurality of story segment 
(section 4.1+). 

Regarding claim 13, Merlino teaches the video stream is communicated from at least 
one of: an analog signal broadcast, a digital signal broadcast, a satellite broadcast, a 
cable broadcast, an Internet connection, a recorder device, and a playback device 
(page 382, section 3+). 

Regarding claim 14, Merlino teaches a storage device (multimedia database- page 386, 
section 401+ and figure 4.1 .1 ) for storing the plurality of story segments. 

Regarding claim 15, Merlino teaches the storage device is a computer memory 
(database of a PC- page 386, section 4.1 +). 
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Regarding claim 16, Zhang teaches at least one of the one or more key frames (news 
items) is a video clip (page 53, line 1+). 

6. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Broadcast News Navigation using Story Segmentation," by Andrew Merlino et al. 
(hereinafter referred to as Merlino), and in view of "Automatic Parsing of News Video," 
by HongJiang Zhang et al (hereinafter referred to as Zhang), and further in view of 
Wactlar et al. (US 5,835,667). 

Regarding claim 12, Merlino in view of Zhang teaches a system as discussed in the 
rejection of claim 1 1 . Merlino further discloses MPEG (page 386, section 4.1 ). However, 
neither Merlino nor Zhang specifically disclose the transform includes a discrete cosine - 
transform. 

Wactlar teaches the transform includes a discrete cosine transform (col. 12, lines 2-32). 
Therefore, it would have been obvious to one of ordinary skill in the art to modify 
Merlino and Zhang to use the teaching as taught by Wactlar in order to improve 
efficiency. 

Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 
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Liu (XP 00062424244) teaches efficient matching and clustering of video shots. 
Ahmad et al. (US 6,263,507) teaches browser for use in navigating a body of 
information, with particular application to browsing information referenced by 
audiovisual data. 

Pullen et al. (US 5,923,376) teaches method and system for the fractal compression of 

data using an integrated circuit for DCT compression/decompression. 

Barton (US 6,327,418) teaches method and apparatus implementing random access 

and time based function on a continuous stream of formatted digital data. 

Warnock et al. (US 6,151,576) teaches mixing digitized speech and text using reliability 

indices. 

8. THIS ACTION IS MADE FINAL, Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Son P Huynh whose telephone number is 703-305- 
1889. The examiner can normally be reached on 8:00-5:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Faile can be reached on 703-305-4380. The fax phone number for 
the organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the customer service office whose telephone number 
is 703-306-0377. 




Son P. Huynh 
December 1 1 , 2003 



ANDREW FAILE 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



